Message-oriented middleware

Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating system and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.[1]

MOM provides software elements that reside in all communicating components of a client/server architecture and typically support asynchronous calls between the client and server applications. MOM reduces the involvement of application developers with the complexity of the master-slave nature of the client/server mechanism.

Contents

Origins

A requirements story

The bank had stored all its customer details on its large mainframe since the 1960s. This mainframe remained in heavy use and had undergone several upgrades.

Although groundbreaking in its day, the mainframe's usefulness to the bank’s staff diminished as the bank introduced new, separate applications based on personal computers (PCs), allowing the bank’s staff to offer customers new services that the mainframe could not support.

An ideal situation would allow the PC-based applications to link to the older mainframe application and allow the mainframe and the PCs to share each other's data. Accessing the mainframe’s data offers two advantages:

  1. new front-end PC applications can replace the old user-unfriendly mainframe terminals
  2. PC-based systems can use the data from the mainframe in new ways — previously impractical due to the constraints of the mainframe’s software

Up until the late 1980s system integrators had no easy way to link these different applications together. Developers faced several challenges:

  1. the developers would have to construct a separate software ‘adapter’ on each system to translate data from source applications into a format that the destination system could understand (and vice versa).
  2. the processing speed of each system would constrain the other system. For example, if the mainframe ran slowly, the PC-based application would have to wait until the mainframe caught up, thereby slowing down the PC application. Conversely, processing that had been offloaded to distributed servers for cost reasons would run slowly and the mainframe would have to wait until the server caught up.
  3. communications programmers would need to install a network gateway system to form a bridge between the mainframe’s network and the PC network if the different systems used different network protocols. The gateway would translate the network packets from the source system and pass them on to the destination system using the destination system’s protocol.

Such issues made integration between applications difficult. Much of such integration also required re-engineering every time two applications on disparate platforms needed to be linked together, as every situation differed to some extent. By devoting effort to linking together applications on different systems, IT departments started to spend an amount significantly greater than that spent on original development per sub-system.

Developers needed a separate piece of software that would sit in the middle of two or more applications and would handle all the ‘plumbing’ between the two systems. Such software needed the intelligence to handle different platforms, different programming languages, various network protocols and diverse hardware. Developers wanted to remove themselves from the complexities of the underlying computing infrastructure so that they could focus on functionality within actual applications.

Towards the end of the 1980s middleware began to emerge that attempted to address these issues. Initial middleware offerings addressed specific handfuls of platforms or languages and thus had limited usefulness. Over time, however, middleware products have become more and more advanced, supporting multiple platforms, languages and protocols.

The ability of middleware to link together disparate systems across a heterogeneous network environment offers only one example of the benefits of this dominant technology. Middleware as of 2006 provides a whole raft of new functionality that augments and enhances the existing applications that it interconnects.

Advantages

Central reasons for using a message-based communications protocol include its ability to store (buffer), route, or transform messages while conveying them from sender to receiver(s).

Synchronicity

MOM comprises a category of inter-application communication software that generally relies on asynchronous message-passing, as opposed to a request-response metaphor. In asynchronous systems, message queues provide temporary storage when the destination program is busy or not connected. In addition, most asynchronous MOM systems provide persistent storage to back up the message queue. This means that the sender and receiver do not need to connect to the network at the same time (asynchronous delivery), and solves problems with intermittent connectivity. It also means that should the receiver application fail for any reason, the senders can continue unaffected, as the messages they send will simply accumulate in the message queue for later processing when the receiver restarts.

Routing

Many message-oriented middleware implementations depend on a message queue system. Some implementations permit routing logic to be provided by the messaging layer itself, while others depend on client applications to provide routing information or allow for a mix of both paradigms. Some implementations make use of broadcast or multicast distribution paradigms.

Transformation

In a message-based middleware system, the recipient's message need not replicate the sender's message exactly. A MOM system with built-in intelligence can transform messages en-route to match the requirements of the sender or of the recipient[2]. In conjunction with the routing and broadcast/multicast facilities, one application can send a message in its own native format, and two or more other applications may each receive a copy of the message in their own native format. Many modern MOM systems provide sophisticated message transformation (or mapping) tools which allow programmers to specify transformation rules applicable to a simple GUI drag-and-drop operation.

Disadvantages

The primary disadvantage of many message oriented middleware systems is that they require an extra component in the architecture, the message transfer agent (message broker). As with any system, adding another component can lead to reductions in performance and reliability, and can also make the system as a whole more difficult and expensive to maintain.

In addition, many inter-application communications have an intrinsically synchronous aspect, with the sender specifically wanting to wait for a reply to a message before continuing (see real-time computing and near-real-time for extreme cases). Because message-based communication inherently functions asynchronously, it may not fit well in such situations. That said, most MOM systems have facilities to group a request and a response as a single pseudo-synchronous transaction.

Lack of standards

The lack of standards governing the use of message oriented middleware has caused problems. All the major vendors have their own implementations, each with its own application programming interface (API) and management tools.

The Java EE programming environment provides a standard API called JMS (Java Message Service), which is implemented by most MOM vendors and aims to hide the particular MOM API implementations; however, JMS does not define the format of the messages that are exchanged, so JMS systems are not interoperable. Microsoft's MSMQ doesn't support JMS, although there are third-party products that can offer this. WebSphere Message Broker, from IBM, does provide JMS support, as well as a whole suite of modern functionality.

The Advanced Message Queuing Protocol (AMQP) is an emerging standard that defines the protocol and formats used in the messaging server and client, so implementations are interoperable. AMQP is defined to provide flexible routing, including common messaging paradigms like point-to-point, fanout, publish/subscribe, and request-response. It also supports transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support. Java applications that use AMQP are typically written in Java JMS. Other implementations provide APIs for C#, C++, PHP, Python, Ruby, and other languages.

Trends

AMQP has been gaining adoption in applications that need an interoperable protocol for Message-oriented middleware.

Other protocols used for message-oriented middleware include XMPP and Streaming Text Oriented Messaging Protocol.

Message-oriented messaging protocols under development include RestMS, a protocol similar in nature to AMQP but constructed over a RESTful HTTP transport, and SPB, a minimalist message framing protocol that can be used to carry higher-level MOM protocols.

An additional trend sees message-oriented middleware functions being implemented in hardware - usually FPGAs or other specialized silicon chips.[3]

See also

References